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(57) Abstract 



A teleconference system (200) is disclosed for digitally recording and playing a conference telephone call that includes a plurality of 
intervals. The teleconference system includes a skim server (55) that detects a first set of die plurality of intervals and a conference bridge 
(100) that detects a seccmd set of die plurality of intervals during die conference call. An interval database server (65) generates labeled 
int^al data for all detected intervals and stores the labeled interval data in a database. The labeled interval data includes an int^al data 
element that defines each interval. After the conference call is recorded, the labeled interval data can be searched and retrieved based 
on assorted criteria. Portions of the recorded conference call associated witfi die retrieved labeled interval data can also be retrieved and 
played back. This facilitates easy retrieval and playback of desired portions or a recorded conference call. Further, during playback of die 
conference call (85), a user interface is generated. The user interface displays the stored labeled interval data. A user can easily select or 
skip to desired portions of the conference call by selecting portions of die user interface! 



FOR THE PURPOSES OP INPORMATiON ONLY 



Codes used to identify States party to the PCT cm the front pages of pamphlets publishmg intmiational applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho^ 


SI 


Slovenia 


AM 


Armenia 


n 


Fmland 


LT 


Lithuania 


SK 


Slovaida 


AT 


Austria 


FR 


France 


LV 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azobaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Cbad 


BA 


Bosnia and Herapgovina 


GB 


Georgia 


MD 


RepobHc of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


T^pkistan 


BE 


Belghini 


GN 


Guinea 


MK 


Hie Conner Yi^gpsbv 


TM 


'Hiilcnuni^an 


BP 


BnilciaaFaso 


GR 


Greece 




Rcpid)Iic of Macedonia 


TR 


Tos/kty 


BG 


Bulgaria 


HU 


Hnqgaiy 


ML 


MaU 


TT 


TVmidad and Tobago 


BJ 


Benin 


IB 


Belaud 


MN 


Mbogoita 


UA 


Ukraine 


BR 




IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United States of Aneiica 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Repnldic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KB 


Kenya 


NL 


Netfaerlancb 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzatao 


NO 


Norway 


ZW 


ZinibabwB 


a 


CAtedlvoira 


KP 


Ocnocratic People's 


NZ 


New Zealand 






CM 


C^meiooD 




Repid>Sc of RoBta 


PL 


BDland 






CN 


Ouna 


ICR 


RepflbHc of Korea 


FT 


Pomigal 






CU 


Cuba 


KZ 


Kazalcstan 


RO 


Romania 






CZ 


Czech Republic 


LC 


.Saint Locia 


RU 


Russian Fedentun 






DE 


Germany 


U 


\ fipchyffnslfffn 


SD 


Sudan 






DK 


Demnart 


LK 


Sri Lanka 


SB 


Sweden 






BB 


Estonia 


LR 


Liberia 


SG 


Sotgapore 







wo 99/17235 



PCT/US98/20446 



METHOD AND APPARATUS FOR STORING AND RETRIEVING 
LABELED INTERVAL DATA FOR MULTIMEDIA iRECORDINGS 



10 : FIELD OF THE INVEKTION 

The present invention is directed to storage and retrieval of 
multimedia data. More particularly, the invention is directed to storage 
and retrieval of labeled interval data in a database. 

15 BACKGROUND OF THE INVENTION 

Unlike records of written communications, records of speech 
communication are rarely recorded, let alone stored, even though storage 
of digital speedi may be readily achieved. It is presently feasible to store 
gigabytes and even terabytes of digitally recorded speech or other types of 

20 multimedia information (e.g., video). Other than for archival purposes, 

there is no practical reason for storing such data without having a 
mechanism by which a user can identify and retrieve only those portions of 
the stored data, which may be of interest. 

The difficulty inherent in seardiing and retrieving digital speech 

25 records stored m a database stems from tfie traditional iqyproaches to 

querying a database to locate particular records. Most database queries are 
logical queries based upon the presence or absence of specified 
characteristics in the records being searched. Boolean logic and fiizzy 
logic have been used to iiK^rease the utility of database queries, but these 
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techniques merely extend the fundamental basis of most typical database 
queries, whether one or more terms, indices, or other identifying 
characteristics are present (or absent) in the records being searched. 

Digital speech records* without being converted into text by speech- 
5 to-text conversion or transcription or otherwise parsed cannot be located 

and/or identified using traditional database query techniques as it is not 
practical to determine whether a word (or phrase) appears in a selected 
portion of recorded speech. Therefore, review of non-transcribed digital 
speech records is frequently limited to listening to the digitally recorded 

10 speech until the item or items of interest are heard. Unfortunately, this 

frequently requires listening to a considerable degree of extraneous or 
irrelevant speech which can be extremely time-consuming without 
providing any significant elucidation. Moreover, digital speech records 
frequently contam lengthy pauses smd, if the digital q^eech r^ 

15 between more than two speakers, it is frequratly difficult, if not 

impossible, to identily the speakers, further exacerbating the problem of 
identifying a specific segment in recorded digital speech. 

Even when a digital speech record is divided into separate digital 
recordings, and each recording is individually accessible and identified, the 

20 digitally recorded data is of limited use. For example, if ten conference 

calls were recorded in a digital storage medium, a user might be able to 
locate a particular conference call on a particular date, if the user were 
fortunate enough to know that the information he or she sought was in that 
specific conference call. Even, so, the user would still have to listm to the 

25 entire recording of the conference call. For a user seeking to identify a 

specific comment made by a specific participant to the conference call, it is 
extremely inefficient for the user to have to listen to the entire conference 
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call. Moreover, if the user does not know the specific date and time of the 
conference call in which the person spoke, the user might have to listen to 
several conference call recordings before finding the desired information. 
Clearly, as soon as a greater than minimal number of recordings were 
5 stored, it becomes inq)ractical for a user to locate desired information 

merely by listening to the conference call recordings. 

Based on the foregoing, there is a need for a method and apparatus 
for readily idemifying, locating, and retrieving stored digital speech and 
other digital multimedia records. 

10 

SUMMARY OF THE INVENTION 

One embodiment of the present invention is a teleconference system 
for digitally recording aiui playing a conference telephone call that im^ludes 
a plurality of intervals, the teleconference system includes a skim server 

15 that detects a first set of the plurality of intervals and a conference bridge 

that detects a second set of the plurality of intervals during the conference 
call. An interval database server generates labeled interval data for all 
detected intervals and stores the labeled interval data in a database. The 
labeled interval data includes an interval data element that defines each " 

20 interval. After the conference call is recorded, flie labeled interval daia can 

be searched and retrieved based on assorted criteria. Portions of the 
recorded conference call associated with the retrieved labeled interval data 
can also be retrieved and played back. This facilitates easy retrieval and 
playback of desired portions of a recorded confi»eiK:e call. 

25 Further, during playback of flie conference call, a user interface is 

generated. The user interface displays the stored labeled interval data. A 
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user can easily select or skip to desired portions of the conference call by 
selecting portions of the user interfiace. 

BRffiF DESCRIPTION OF THE DRAWINGS 
S Fig. 1 illustrates a teleconference syst^ in accordance with one 

embodiment of the present invration. 

Fig. 2 illustrates the format of an interval data element that forms 
the labeled interval data associated with a recorded conference. 

Fig. 3 illustrates a conference playback document in accordance 
10 with one embodiment of the present invention. 

Fig. 4 illustrates in detail how overlapping intervals are displayed. 



DETAILED DESCRIPTION 

15 In one embodiment of the present invention, intervals within 

recorded digital speech or other multunedia data are specffically identified 
and labeled. The labeled interval data provides a mechanism by which a 
user can specifically identify an interval within digitally recorded 
multimedia, and having identified that interval, retrieve it and other 

20 intervals sharing desked characteristics. 

Fig. 1 illustrates a teleconferCTce system in accordance with one 
embodunent of the present invention. Teleconference system 200 records 
and stores a teleconference call and associated labeled interval data. 
Teleconference system 200 further allows a recorded teleconference to be 

25 played back using the stored labeled int^al data. 
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The main components of teleconference system 200 are a 
conference recorder 1 10, a skim server 55, an interval database ("IDB") 
server 65, and a Java user interface 85. 

In teleconference system 200, a pIuraliQr of telephones 31, 32, and 
5 33 aire interconnected through the public switched telephone network 

CPSTN'')40. One or more individuals may participate m a teleconference 
through each telephone 31-33. The particq)ants may be identified by the 
telephone they are calling from or, alternatively, by voice recognition or 
other forms of identification during the teleconference. 

10 A teleconference may be initiated by a conference host accessing a 

WebRoom interfece on a WebRooms server 50. A WebRoom mterface 
provides a mechanism by which participants may be actively added to 
and/or deleted from a teleconference. In one embodunent, the WebRoom 
mterface for all teleconference participants is implemented as Common 

15 Gateway Interfece ("CGD program 60 on an HyperText Transport 

Protocol Web Server ("Httpd") 70 that provides interactive control of the 
teleconfereiice through Hyper-Text Markup Laiig^ 
documents. The HTML documents are accessible as conference pages 80 
through a Web browser 90 such as Netscape® Navigator or Int^net 

20 E3q)lorer®. 

At record time, the conference host uses WebRooms server 50 to 
dial a conference scribe. The conference scribe acts as an additional 
participant to the teleconference. At the same tinne, conference recorder 
110 tells IDB Server 65 to create a new collection pomt, referred to as a 

25 "depot" for storing aU data related to this particular recordmg, and it tells 

skun server 55 to begin recording an audio file using, for exanq)le, a 
Dialogic board 57 from Dialogic Corp., or its equivalent. A depot in 
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teleconference system 200 can be a structured query language ("SQL'') 
database 35 coupled to an Open DataBase Connectivity ('•ODBC) 
interface 36. While the conference is running, conference bridges 100 
detects call control events (e.g. , which participant is talking, new 

5 particq)ants being added, etc.; and sends fh^ events through WebRooms 

server 50 and conference recorder 110 into flie new depot (i.e., SQL 
database 35). Meanwhile, skim server 55 detects pauses in speech and 
adds these events as well to the depot. The events detected by both 
conference bridges 100 and skim server 65 are referred to as "intervals". 

10 When playing back a recorded conference on teleconference system 

200, the user brings up a Java user interface 85 to select a recording 
accessed via IDB server 65. The user interface 85 retrieves labeled 
interval data for the recording and uses them to display a visual time-line 
of events. The user enters a phone number that is passed to Skim Server 

15 55 so it can call the user's telephone for conference playback through 

Dialogic board 57. As the audio plays on the user's phone, Java user 
interface 85 continuously updates the graphical display and controls how 
the recording is played using skim server 55. All clients like Java user 
mterface 85 and conference recorder 110 communicate with skun server 55 

20 and IDB server 65 through a CORBA application programming mter£lce in 

one embodiment of the present invoition. CORBA was chosen because it 
allows a simple interface between programs written in different languages 
running on different platforms. Both servers 50 and 55 and conference 
recorder 110 are written m C++ and run on Sun Solaris platforms in one 

25 embodiment of the present invention. 

Skim server 55 performs the following functions: 
1 . Records audio from telephone Ime to file. 
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2. Detects speech events while recordmg and posts them to the 
database. 

3. Plays from file to telephone line 

- from any point in recording 
S - in variable speeds 

- with pauses removed or not. 

In one embodiment, skim server 55 is based on the same type of 
hardware as standard voice mail servers, and it performs many of the same 
functions. One difference between skim server 55 and a more traditional 

10 voice mail server is that it processes speech events and posts them to IDB 

server 65, and also that it provides fine control over what parts of the 
audio file are played and what parts are skipped. 

One fimction of IDB server 65 is to store and retrieve labeled 
mterval data associated with a recorded conference. This is data diat 

15 describes pix)peities about specific intervals within the speech, such as who 

is talking, pauses in speech, telephone call control data, etc. This can be 
further extended to applications that require intervals that mark video scene 
changes, or relate automatic speech recognition ou^ut to a recording. The 
labeled interval data can be created, stored, and retrieved by a number of 

20 different plications. Some are automatically derived from raw speech 

data, some are side effects of user activity, and others may be entered 
manually at record time or at playtime. 

Fig. 2 illustrates the format of an mterval data element 130 that 
forms die labeled interval data associated witii a recorded conference. 

25 Every interval during the recorded conference will be associated with an 

interval data element 130. In one embodiment, each interval data element 
130 includes the following: 
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1. Recording ID or Depot 122: Refers to the recording that is 
associated with the interval and the collection point where 
the recording is stored. 

2. Start time 123: Applications need both absolute time and 

5 time relative to recording start time. Relative time is more 

conq)act, and it is easy to convert to absolute as long as an 
absolute start time is stored with the recording. 

3. Duration or end time 124. 

4. Type: A code to identify the meaning of this interval. Is it 
10 a pause m speech^ a scene change, etc.? 

5. Type-specific data values 126: Depending on the type, this 
data could be a string of text, a number, a URL, etc. 

Labeled interval data must be able to be stored, retrieved, and 
manipulated more than one at a time. Some applications will deal with 

15 large collections of intervals that share everything except start time and end 

time (e.g. , all times when a specific person was speaking). 

Applications must be able to store interval data in the database at 
any thne: before recording has begun, during recording, and after. For 
example, for a teleconference it may be necessary to record caller-id and 

20 ringing events before the call, record who is spealdng during the call, and 

make annotations about the call afterwards. Some applications need to 
display inconq)lete interval data while a recording is in progress (e.g., 
catch l^> to live conference), so it should be possible to post an interval that 
has started but not ended yet, and post the end time later. It should also be 

25 possible to adjust interval data, for example to realign tfiem with other 

data. 
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All applications that post events to IDB server 65 must specify 
precise millisecond offsets for start and end times of each interval. All 
offsets are from an absolute start-time for the recording. Posting intervals 
from different machines in real-time requires all clients that are posting 
events have synchronized clocks, so standard network time protocol 
("NTP") software is run on all of these machines. 

Browse, search, and playback applications need to query and 
display subsets of interval data. Exanq>les of queries that can be supported 
by the present invention include: 

• All interval data for a specific recording, sorted by time and 
type. 

• All intervals of a specific typt with specific values, or 
values withm a particular range* 

• Intervals within an absolute or relative time range. 
15 • luteals of a specific duration. 

The present invention provides for logical/set operations. For 
example, assume a user wants to see and/or hear only the parts of a 
recording when person A or person B was talking, and wants to leave all 
the pauses out. This can be ei^ressed by making three queries: intervals 

20 when A was speaking (set A), mtervals when B was speaking (set B), and 

pause intervals (set P). The desired set can be caressed as "A union B 
less P", or if these sets are thought of as long bit masks, then they can be 
described as logical operations: (A B)&( P). 

Some types of mtervals may not have clear start and end times. 

25 Instead of a binary on/off state at each time increment, some data has an 

associated probability curve over time because the exact times of the events 
are not certain. Output from automatic speech recognition (e.g., phoneme 
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lattices) can include several overlapping hypotheses about what words are 
being said at any given moment. In one embodiment of the present 
invention, IDB server 65 provides siq>port for ''fiizzy" intervals. In 
another embodiment, IDB server 65 uses binary intervals aloi^ witfi a 
5 probabiliQr vahie in the ^e-specific numeric data field to achieve a similar 

effect as fiizzy intervals, but without fuzzy logical operations. 

Transcriptions can be stored as interval data, perhaps one sentence 
per interval, or one word per interval depending on how fine a inapping is 
desired between words and time. The transcriptioiis may be produced from 

10 close caption text, higher quality ofT-line transcriptions, or a lower quality 

automatic speech recognition system. 

Teleconfereiice system 200 provides playback of recorded 
conferences using conference playback documents, The system utilizes 
stored labeled mterval datai associated with the conference. Fig. 3 

15 illustrates a conference playback document 300 in accordance with one 

embodiment of the present invention. Conference playback document 300 
is implemented as a Java applet through Java user interface 85 of Fig. 1 . It 
uses a visual structuring of the recording as a series of color-coded 
intervals (e.g., intervals 305 and 310) plotted on a horizontal thne axis in 

20 an area referred to as a tfane-line window 315, Each partic^ant in a call 

(e.g., particq)ants 316-320) is allocated a separate time-line for graphically 
depicting all labeled intervals that are associated with that pcfrson (e.g., 
dialing, connected, muted, talking, etc.). 

Fig. 4 illustrates in detail how overlapping intervals are displayed. 

25 As shown in Fig. 4, by plotting each interval type one at a time, starting 

with taller bars, the document displays overlapping intervals on the same 
line. 
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Refeiring again to Fig* 3, intervals tbat are not associated with an 
individual person are plotted separately above the participants, (e.g., 
hyperlinks 330, speech segments, etc.). Time-line window 315 provides a 
snapshot of every participants' activity, and can be used to navigate 
5 through die recording. 

In one raibodiment, once users have established a phone connection 
to the recorded conference player, they can use a tool bar 350 below the 
time-line to begin playmg the audio and adjust the skmunmg parameters. 
In another embodiment, a separate phone connection is not necessary 
10 because the audio conference recording can be "streamed" in conjunction 

with conference playback document 300. 

Toolbar 350 provides five buttons to control the player: **goto 
beginning 351", "jump back 352", "stop 353", "play 354", and "jump 
forward 355". It also contains a slider 356 for adjusting the playmg speed 
15 (0.7x, 1 .Ox, l,3x, 1 .7x, and 2.0x), a zoom menu 357 for selecting the 

zoom factor (hone, 20min., lOmin., and 5min.), and an on/off pause 
button 358 for pause removal. 

As the recorded conference audio plays, a vertical red needle 360 
moves across tfie time-line. When needle 360 moves, every participant's 

20 name tag is colored to reflect that person's state at that time m the meeting. 

Fig. 3 shows a one hour conference with the entire duration visible (zoom 
= none). In this view, the visual structures help make some details of the 
call immediately obvious. For example, die number and span of tiie light 
colored bars can identify the most/least dominant talkers. The initial long 

25 umntemipted talking bands show who gave the formal presentations. 

Rnally die point where the question and answer session began is visible 
roughly half way into the call, where many short talking intervals are 



wo 99/17235 



12 



PCf/tJS$8AZ0446 



scattered amoiog many participants. More detailed information must be 
foimd by either listening to the audio or by searching through linked 
annotations, images, and other documents. 

The zooming feature allows the user to narrow the duration 
5 displayed in the time-lme window. A numbered scroll bar allows tibe user 

to register the zoomed-in portion with the full duration, and scroU using 
mouse clicks or arrow keys on the keyboard. Scrollmg is independent of 
player location needle 360, so the user can separately glance at regions, 
without disruptmg listening. Player needle 360 can be moved by clicking 

10 on Ae time-line, or by pressmg a jump forward/backward button. When 

this happens, the skim server plays a short non-speech audio cue and 
begins to play at the new location. 

Clicking the time-line near the top is used to select hyperlinks (e.g. , 
link 330) rather than to move the needle. When a link is selected, or a 

15 "links" button 340 is pressed, a dialog displays all the links in the 

recording. This dialog can be used to visit a Imk, edit a link, or create a 
link both in and out of the tune-line. One embodiment of the present 
invention supports the following types of links: annotations, audio, 
documents, images, and general URL. All links are implemented using 

20 URLs except annotations, which store textual content as interval data. 

Each type of link is displayed on die time-lme with a representative icon. 

Hyperlinks into and out of the time-line are stored as intervals, and 
contain both a beginning and endmg time offset. Thus a Unk can refer to a 
particular pomt or region of the time-line, allowing a rich set of simTmiitig 

2S alternatives. For exsanplt, following a luik can cause play to begin at a 

certain point, end at a certain point, or sequence through selected regions. 
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This means tbat following a link can have multiple effects, inchiding 
moving the player needle and changing the document page. 

As disclosed, one mibodiment of the present invention is a 
teleconference recorder and player. When a conference is recorded, an 
5 interval database stores labeled interval data associated with the 

conference. The labeled interval data allows searching and retrieving of 
the recorded conference, and facilitates playback of the recorded 
conference. 

Several embodiments of the present invention are specifically 
10 illustrated and/or described herein. However, it will be appreciated that 

modifications and variations of the present invention are covered by the 
above teachings and within the purview of the appended claims without 
departing firom the spirit and intended scope of the invention. 

For example, although the embodiments disclosed are implemented 
15 over the Internet, the present invention can be implemented using a private 

network, or using any other known or future data communication methods. 
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1 WHAT IS CLAIMED IS; 

2 1. A system for recording aid playing midtimedia data 

3 mcludes a plurality of intervals, said system conq)rising: 

4 asldmserverdiatdetectsafirstsetof the plurality of intervals; 

5 an interval database server coi^led to said skim server, said interval 

6 database server generating labeled interval data for the first set of the 

7 plurality of intervals detected by said skim s^er; and 

8 a database coiQ)led to said interval database server and storing said 

9 labeled interval data; 

10 wherein said labeled interval data comprises an interval data 
U element for each of tiie detected plurality of intervals. 

12 2. The system of claim 1 , further comprising: 

13 a conference bridge coupled to said interval database server that 

14 detects a second set of the phirality of uitervals; 

15 wherein said interval database server furdier generates labeled 

16 interval data for the second set of the plurality of intervals detected by said 

17 skim server. 

18 —3. Thesyst)emofclami2,wherem said first set of the plurality of 

19 intervals comprise pauses in speech. 

20 4. The system of claim 2, wherein said second set of the plurality 

21 of intervals conq>rise call control events. 

22 S. The system of claim 1, wherein the multimedia data comprises a 

23 conference telephone call. 

24 6. Thesystemofclahnl, wherein said mterval data element 

25 conq)rises: 

26 a type of the detected interval; 
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27 a Start time of the detected interval; and 

28 a duration of the detected interval. 

29 7. The system of claim 6» wherein said interval data element 

30 fiuHier comprises: 

31 a recording identification of the detected interval; and 

32 atype-spedficdata value of die detected interval. 

33 8. The system of claim 1, wherein said interval database server 

34 comprises: 

35 means for searching said stored labeled interval data. 

36 9. The system of claun 8, wherein said interval database server 

37 further conq)rises: 

3S means for retrieving said stored labeled interval data and associated 

39 multimedia data. 

40 10. The system of claim S, further comprising: 

41 a user mterfacegenen^ during playback of the conference call, 

42 wherein said user interface displays die stored labeled interval data. 

43 11. A method for recording and playing multimedia data that 

44 includes a plurality of intervals, said method conq)rising:. 

45 detectmg die plurality of intervals; 

46 generating labeled interval data for the plurality of intervals; and 

47 storing the labeled interval data in a database; 

48 wherein said labeled interval data comprises an interval data 

49 elmient associated with each of the plurality of intervals. 

50 12. The method of claim 11, wherein said interval data element 

51 coii4)ri5es: 

52 a type of the associated interval; 

53 a start time of the associated interval; and 
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54 a duration of flie associated interval. 

55 13. The method of claim 12» wherein said interval data element 

56 further con^rises: 

57 a recording identification of the associated interval; and 

58 a type-specific data value of the associated interval. 

59 14. The method of claim 11, further conq)rising: 

60 storing the multimedia data in the database. 

61 IS. The method of claim 14, further comprising: 

62 queryiag said database based on one or more labeled interval data 

63 parameters; and 

64 retrieving at least one interval data element and associated 

65 multimedia data from the database. 

66 16. The method ofclaim 11, wherein the multimedia data 

67 comprises a conference telephone call. 

68 17. The method of claim 16, fiirther con^rising: 

69 generating a user inter&ce Oat displays the kbeled m^^ 

70 and 

71 playing die confidence call based on selections of the user inter&ce. 

72 18. A method of recording and playiog a teleconference telephone 

73 call, said method conq)rising: 

74 detecting a plurality of intervals during the telephone caU; 

75 generatmg labeled interval data for each of said phirality of 

76 intervals; and 

77 storing said labeled interval data in a database. 

7S 19. The method ofclaim 18, iKterein said labeled interval data 

79 comprises a plurality of interval data elements, said method further 

80 comprising: 
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SI querying said database and retrieving one or more of the stored 

82 interval data elements; and 

83 playing a portion of the teleconference telephone call that is 

84 associated with each of said retrieved interval data elements. 

85 20. The mediod of claim 18, wherein said detected intervals 

86 conq)rise: 

87 an identity of a speaker; 

88 pauses in speech; and , 

89 telephone call control. 
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